When a human communicates with a machine using natural language on the web and online, how can it understand the human's intention and semantic context of their talk? This is an important AI task as it enables the machine to construct a sensible answer or perform a useful action for the human. Meaning is represented at the sentence level, identification of which is known as intent detection, and at the word level, a labelling task called slot filling. This dual-level joint task requires innovative thinking about natural language and deep learning network design, and as a result, many approaches and models have been proposed and applied. This tutorial will discuss how the joint task is set up and introduce Spoken Language Understanding/Natural Language Understanding (SLU/NLU) with Deep Learning techniques. We will cover the datasets, experiments and metrics used in the field. We will describe how the machine uses the latest NLP and Deep Learning techniques to address the joint task, including recurrent and attention-based Transformer networks and pre-trained models (e.g. BERT). We will then look in detail at a network that allows the two levels of the task, intent classification and slot filling, to interact to boost performance explicitly. We will do a code demonstration of a Python notebook for this model and attendees will have an opportunity to watch coding demo tasks on this joint NLU to further their understanding.

We propose a PiggyBack, a Visual Question Answering platform that allows users to apply the state-of-the-art visual-language pretrained models easily. The PiggyBack supports the full stack of visual question answering tasks, specifically data processing, model fine-tuning, and result visualisation. We integrate visual-language models, pretrained by HuggingFace, an open-source API platform of deep learning technologies; however, it cannot be runnable without programming skills or deep learning understanding. Hence, our PiggyBack supports an easy-to-use browser-based user interface with several deep learning visual language pretrained models for general users and domain experts. The PiggyBack includes the following benefits: Free availability under the MIT License, Portability due to web-based and thus runs on almost any platform, A comprehensive data creation and processing technique, and ease of use on deep learning-based visual language pretrained models. The demo video is available on YouTube and can be found at https://youtu.be/iz44RZ1lF4s.

协作过滤问题通常是基于矩阵完成技术来解决的，该技术恢复了用户项目交互矩阵的缺失值。在矩阵中，额定位置专门表示给定的用户和额定值。以前的矩阵完成技术倾向于忽略矩阵中每个元素（用户，项目和评分）的位置，但主要关注用户和项目之间的语义相似性，以预测矩阵中缺少的值。本文提出了一种新颖的位置增强的用户/项目表示培训模型，用于推荐，Super-Rec。我们首先使用相对位置评级编码并存储位置增强的额定信息及其用户项目与嵌入的固定尺寸，而不会受矩阵大小影响。然后，我们将受过训练的位置增强用户和项目表示形式应用于最简单的传统机器学习模型，以突出我们表示模型的纯粹新颖性。我们对建议域中的位置增强项目表示形式进行了首次正式介绍和定量分析，并对我们的Super-Rec进行了原则性的讨论，以表现优于典型的协作过滤推荐任务，并具有明确的和隐式反馈。

在将文档解析为下游应用程序的结构化，机器可读格式时，识别非结构化数字文档的布局至关重要。文档布局分析中的最新研究通常依靠计算机视觉模型来理解文档，同时忽略其他信息，例如上下文信息或文档组件的关系，这对于捕获至关重要。我们的DOC-GCN提出了一种有效的方法，可以协调和整合异质方面以进行文档布局分析。我们首先构造图形以明确描述四个主要方面，包括句法，语义，密度和外观/视觉信息。然后，我们应用图形卷积网络来表示信息的各个方面，并使用池进行集成。最后，我们将各个方面汇总，并将它们送入2层MLP，以进行文档布局组件分类。我们的DOC-GCN实现了新的最先进的结果，从而获得了三个广泛使用的DLA数据集。

注意机制已被用作跨视觉和语言（VL）任务的重要组成部分，以弥合视觉和文本特征之间的语义差距。尽管注意力已被广泛用于VL任务，但尚未研究其在弥合视觉和文本线索之间语义差距方面的不同注意对准计算的能力。在这项研究中，我们通过研究注意力评分计算方法，并检查其实际代表视觉区域的作用以及文本令牌对全球评估的重要性，对了解注意力对齐的作用进行全面分析。我们还分析了注意力分数计算机制的条件更多（或更少）可解释，并且可能会影响三个不同VL任务的模型性能，包括视觉问题答案，文本到图像生成，文本和图像匹配（句子和图像检索）。我们的分析是同类产品中的第一个，并提供了在VL任务的训练阶段应用的每个注意力对齐得分计算的重要性，这通常在基于注意力的交叉模态模型和/或预审前的模型中被忽略。

为了防止青年自杀，社交媒体平台受到了研究人员的广泛关注。一些研究应用机器学习或基于深度学习的文本分类方法来对包含自杀风险的社交媒体帖子进行分类。本文复制了基于社交媒体的自杀性检测/预测模型。我们评估了使用多个数据集和不同最先进的深度学习模型（RNN-，CNN-和基于注意力的模型）检测自杀构想的可行性。使用两个自杀性评估数据集，我们通过定量和定性方式评估了28种输入嵌入的组合和4种常用的深度学习模型和5种预处理的语言模型。我们的复制研究证实，深度学习总体上可以很好地适用于基于社交媒体的自杀性检测，但这在很大程度上取决于数据集的质量。

Process monitoring and control are essential in modern industries for ensuring high quality standards and optimizing production performance. These technologies have a long history of application in production and have had numerous positive impacts, but also hold great potential when integrated with Industry 4.0 and advanced machine learning, particularly deep learning, solutions. However, in order to implement these solutions in production and enable widespread adoption, the scalability and transferability of deep learning methods have become a focus of research. While transfer learning has proven successful in many cases, particularly with computer vision and homogenous data inputs, it can be challenging to apply to heterogeneous data. Motivated by the need to transfer and standardize established processes to different, non-identical environments and by the challenge of adapting to heterogeneous data representations, this work introduces the Domain Adaptation Neural Network with Cyclic Supervision (DBACS) approach. DBACS addresses the issue of model generalization through domain adaptation, specifically for heterogeneous data, and enables the transfer and scalability of deep learning-based statistical control methods in a general manner. Additionally, the cyclic interactions between the different parts of the model enable DBACS to not only adapt to the domains, but also match them. To the best of our knowledge, DBACS is the first deep learning approach to combine adaptation and matching for heterogeneous data settings. For comparison, this work also includes subspace alignment and a multi-view learning that deals with heterogeneous representations by mapping data into correlated latent feature spaces. Finally, DBACS with its ability to adapt and match, is applied to a virtual metrology use case for an etching process run on different machine types in semiconductor manufacturing.

Dynamic treatment regimes assign personalized treatments to patients sequentially over time based on their baseline information and time-varying covariates. In mobile health applications, these covariates are typically collected at different frequencies over a long time horizon. In this paper, we propose a deep spectral Q-learning algorithm, which integrates principal component analysis (PCA) with deep Q-learning to handle the mixed frequency data. In theory, we prove that the mean return under the estimated optimal policy converges to that under the optimal one and establish its rate of convergence. The usefulness of our proposal is further illustrated via simulations and an application to a diabetes dataset.

Searching long egocentric videos with natural language queries (NLQ) has compelling applications in augmented reality and robotics, where a fluid index into everything that a person (agent) has seen before could augment human memory and surface relevant information on demand. However, the structured nature of the learning problem (free-form text query inputs, localized video temporal window outputs) and its needle-in-a-haystack nature makes it both technically challenging and expensive to supervise. We introduce Narrations-as-Queries (NaQ), a data augmentation strategy that transforms standard video-text narrations into training data for a video query localization model. Validating our idea on the Ego4D benchmark, we find it has tremendous impact in practice. NaQ improves multiple top models by substantial margins (even doubling their accuracy), and yields the very best results to date on the Ego4D NLQ challenge, soundly outperforming all challenge winners in the CVPR and ECCV 2022 competitions and topping the current public leaderboard. Beyond achieving the state-of-the-art for NLQ, we also demonstrate unique properties of our approach such as gains on long-tail object queries, and the ability to perform zero-shot and few-shot NLQ.

Energy storage resources must consider both price uncertainties and their physical operating characteristics when participating in wholesale electricity markets. This is a challenging problem as electricity prices are highly volatile, and energy storage has efficiency losses, power, and energy constraints. This paper presents a novel, versatile, and transferable approach combining model-based optimization with a convolutional long short-term memory network for energy storage to respond to or bid into wholesale electricity markets. We apply transfer learning to the ConvLSTM network to quickly adapt the trained bidding model to new market environments. We test our proposed approach using historical prices from New York State, showing it achieves state-of-the-art results, achieving between 70% to near 90% profit ratio compared to perfect foresight cases, in both price response and wholesale market bidding setting with various energy storage durations. We also test a transfer learning approach by pre-training the bidding model using New York data and applying it to arbitrage in Queensland, Australia. The result shows transfer learning achieves exceptional arbitrage profitability with as little as three days of local training data, demonstrating its significant advantage over training from scratch in scenarios with very limited data availability.